On the internet there are already several search engines for medical images available, such as Yotta or Goldminer, which enable the user to search for images given a keyword. The images or their reference are entered into the system by web crawling or uploading by publishers and users. Subsequently, these images are manually tagged by users or by a heuristic considering the context information at the web site where the image was found. Unfortunately, the automatic approach cannot be that precise as needed for validating diagnosis because of a lack of real understanding of the image. Hence, the system also cannot provide automatic zoom into or navigation to the image location visualizing only the queried anatomical structure.
Another disadvantage is that the user has to load the whole image and scroll to the desired image location, the region of interest (ROI), which is very time-consuming, especially if the user wants to compare ROI from hundreds of patients. In this respect the huge progress in medical image acquisition in the last decade has to be mentioned, which led to images of gigabyte size acquired within seconds, and the trend towards larger images is still ongoing, which has immense impact on image loading time.
As an example, the user wants to find medical images which only show the heart without disturbing image region which draw off attention and then display in parallel with images from several other patients in a gallery, the images need to be adequately shrunken to the ROI to fit on the screen and enable easy comparison by the user.
Another major aspect and disadvantage of known systems is the lack of semantic understanding of the query string itself in current systems. What would be beneficial is that the system works semantically such that a query can be automatically expanded into meaningful related sub-queries to increase recall, e.g., given the heart, the search should also search for images tagged with narrower terms such as aortic valve or myocardial chambers, which are both components of the heart.
Developing new image processing algorithms in industry or academia need to train or evaluate against a huge set of image examples. It is a very time-consuming and incomplete task to find all images showing an adequate ROI needed for the algorithm development. The situation today is that the researcher manually inspects images in a database, crops the relevant part and stores them in his own developer database. There is a huge trend towards trained image algorithms by the use of statistical models coming from machine learning. These algorithms are used to build software detectors which automatically localize an object in a medical image. This approach requires that positive and negative examples are presented to a training algorithm, i.e. image regions showing and explicitly not showing the object which should be learned. Today this is all hand work.
In state of the art systems in medicine users cannot retrieve images from a database shrunken to that part of the image showing the anatomical structure of interest. They have to load the whole image and scroll to that location or crop the ROI by hand. This is extremely time-consuming and impossible if hundreds of patients for a given anatomical structure should be compared in parallel.
Even at the beginning, when the user inputs a query string into a search engine, the system does not really understand what is meant; it just compares texts with patterns. This makes it impossible to expand the query to meaningful, related queries which are from the medical point of view also relevant, e.g., as another example what the user wants: querying for images showing the abdomen should result in images tagged with anatomical structures located in the abdomen such as the liver, the pancreas, the intestine, etc. (these are hundreds or even thousands of items)
The developer of an image processing algorithm needs to crop and label ROI by hand to correctly present the data to a training algorithm. This is a very time-consuming task and must always be redone for every new anatomical structure where such an object detector should be developed.
There is no other solution today than manually checking every medical image in the database, loading the image and cropping the relevant parts. This procedure is extremely time-consuming. Query strings must be expressed with care to match the tags in the database and must be manually expanded into meaningful sub-queries to increase the search precision and recall.